Depth-based surveillance image reconstruction

ABSTRACT

A subject may be interrogated with electromagnetic radiation for producing image data representative of an image of the subject. First and second image data, produced based on reflectivity in first and second frequency ranges, may be related. In some examples, image data for one picture element may be produced based at least in part on reflectivity of the radiation at a depth selected based at least in part on a depth of another picture element. In some examples, plural images may be produced based on reflectivity at different depths. In some examples, a value of reflectivity may be determined for an intermediate depth between two adjacent depths of image data based at least in part on reflectivity of the radiation for a plurality of spaced-apart depths.

BACKGROUND

Millimeter wave signals are used for radar and telecommunications. Theyare also capable of being used to produce data representative of asubject, by directing millimeter-wave signals at the subject anddetecting the reflected signal. The data produced may then be used toproduce an image of the subject. Examples of such imaging systems aredescribed in U.S. Pat. Nos. 5,455,590; 5,557,283; 5,859,609; 6,507,309;and 6,703,964; and U.S. Patent Publication Nos. 2004/90,359;2004/140,924; and 2004/263,379, which patent references are incorporatedherein by reference.

BRIEF SUMMARY OF THE DISCLOSURE

A method for imaging a subject may include, or an imaging system mayprovide, interrogating the subject with electromagnetic radiation, andproducing, from the interrogating, image data representative of at leasta first image of at least a portion of the subject. In some examples,the electromagnetic radiation may include radiation in first and secondfrequency ranges, with first and second image data being produced basedon reflectivity of the electromagnetic radiation in the first and secondfrequency ranges, and the first and second image data being related. Insome examples, first image data may be for a plurality of depths foreach of a plurality of picture elements, with at least second image databeing produced that is representative of at least a first image of atleast the portion of the subject based at least in part on reflectivityof the electromagnetic radiation at a second depth selected based atleast in part on the first depth of at least another one of the pictureelements. In some examples, first image data may be for a plurality ofdepths for each of a plurality of picture elements, with plural imagesproduced based on reflectivity at different depths. In some examples,the first image data may be based at least in part on reflectivity ofthe electromagnetic radiation for a plurality of spaced-apart depths foreach of a plurality of adjacent picture elements, and a value ofreflectivity determined for an intermediate depth between two adjacentdepths.

BRIEF DESCRIPTION OF THE SEVERAL FIGURES

FIG. 1 is a general diagram showing an imaging system.

FIG. 2 is a general diagram showing an example of a surveillance imagingsystem according to FIG. 1.

FIG. 3 is a general flow chart illustrating an example of a method ofoperation of an imaging system of FIG. 1 or FIG. 2.

FIG. 4 is a representative image from an imaging system of FIG. 1 orFIG. 2 using a full frequency band of electromagnetic radiation.

FIG. 5 is a representative image from an imaging system of FIG. 1 orFIG. 2 using a first range of frequencies of electromagnetic radiation.

FIG. 6 is a representative image from an imaging system of FIG. 1 orFIG. 2 using a second range of frequencies of electromagnetic radiation.

FIG. 7 is a representative image from an imaging system of FIG. 1 orFIG. 2 using a third range of frequencies of electromagnetic radiation.

FIG. 8 is an image produced from the images of FIGS. 5–7.

FIG. 9 is the image that is produced from the images of FIGS. 4 and 8.

FIG. 10 is the image of FIG. 9 modified to illustrate changes in theimage of FIG. 4.

FIG. 11 is an image of three-dimensional data space used to representimage data for an interrogated subject.

FIG. 12 is a representative image from an imaging system of FIG. 1 orFIG. 2 produced by using maximum reflectance values for each pictureelement.

FIG. 13 is an image showing depths used for the image of FIG. 12.

FIG. 14 is a chart showing reflectance as a function of depth for threeadjacent picture elements in image data used to produce the image ofFIG. 12.

FIG. 15 is an image produced by using reflectance values existing at acommon first depth corresponding to a maximum reflectance in the chartof FIG. 14.

FIG. 16 is an image produced by using reflectance values existing at acommon second depth corresponding to a maximum reflectance value in thechart of FIG. 14.

FIG. 17 is an image of depths selected in a depth filter algorithm.

FIG. 18 is an image of reflectance values using the depths shown in FIG.17.

FIG. 19A is an image produced by using reflectance values in a firstdepth plane.

FIG. 19B is an image illustrating generally the areas of higherreflectance values in the image of FIG. 19A.

FIG. 20A is an image produced by using reflectance values in a seconddepth plane.

FIG. 20B is an image illustrating generally the areas of higherreflectance values in the image of FIG. 20A.

FIG. 21A is an image produced by using reflectance values in a thirddepth plane.

FIG. 21B is an image illustrating generally the areas of higherreflectance values in the image of FIG. 21A.

FIG. 22 is an image formed as a set of side-by-side images of FIGS. 19A,20A and 21A.

FIG. 23A is an image formed by combining the images of FIGS. 19A, 20Aand 21A.

FIG. 23B is an image illustrating generally the respective contributionsof the three images of FIGS. 19A, 20A and 21A in the image of FIG. 23A.

FIG. 24 is a chart of reflectance value as a function of depth for tworepresentative picture elements and illustrating an interpolationalgorithm.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

There are situations in which it is desirable to identify features of asubject, particularly features of a person and any objects carried bythe person. For example, it may be desired to determine whether thesubject includes objects not apparent from a visual inspection of thesubject. For example, when monitoring people prior to entry into acontrolled-access environment, such as a public, private or governmentfacility, building or vehicle, observations may be facilitated byemploying millimeter-wave imaging technology. Regardless of theapplication, the benefits derived from the monitoring may depend on thespeed and accuracy of the monitoring, and where appropriate, theeffectiveness of identifying visually hidden objects. Techniques andsystems that provide increased accuracy of imaging and distinguishdifferences in related images may be beneficial.

In the description and claims that follow, the terms feature andcharacteristic may be synonymous. For example, intensity, color, depthor distance relative to a reference, and values of intensity, color ordepth may be features or characteristics of image data, and thereby animage or picture element of an image.

Imaging Systems

Shown generally at 30 in FIG. 1 is an exemplary imaging system. System30 may include an interrogating apparatus 32, a controller 34, and insome systems, an output device 36. The system may interrogate a subject38 in the sense that the interrogating apparatus transmitselectromagnetic radiation 40 toward the subject, and in response, thesubject emits or reflects electromagnetic radiation 42 that is detectedby the interrogating apparatus.

Subject 38 may include all that is presented for interrogation by aninterrogation system, whether human, animal, or inanimate object. Forexample, if a person is presented for interrogation, subject 38 mayinclude the entire person's body or a specific portion or portions ofthe person's body. Optionally, subject 38 may include one or morepersons, animals, objects, or a combination of these.

System 30 may be adapted to interrogate subject 38, throughinterrogation apparatus 32, by irradiating it with electromagneticradiation, and detecting the reflected radiation. Electromagneticradiation may be selected from an appropriate frequency range, such asin the range of about 100 megahertz (MHz) to 2 terahertz (THz), whichrange may be generally referred to herein as millimeter-wave radiation.Accordingly, imaging, or the production of images from the detectedradiation, may be obtained using electromagnetic radiation in thefrequency range of one gigahertz (GHz) to about 300 GHz. Radiation inthe range of about 5 GHz to about 110 GHz may also be used to produceacceptable images. Some imaging systems use radiation in the range of 24GHz to 30 GHz. Such radiation may be either at a fixed frequency or overa range or set of frequencies using one or more of several modulationtypes, e.g. chirp, pseudorandom frequency hop, pulsed, frequencymodulated continuous wave (FMCW), or continuous wave (CW).

Certain natural and synthetic fibers may be transparent orsemi-transparent to radiation of such frequencies and wavelengths,permitting the detection and/or imaging of surfaces positioned beneathsuch materials. For example, when the subject of interrogation is anindividual having portions of the body covered by clothing or othercovering materials, information about portions of the subject's bodycovered by such materials can be obtained with system 30, as well asthose portions that are not covered. Further, information relative toobjects carried or supported by, or otherwise with a person beneathclothing can be provided with system 30 for metal and non-metal objectcompositions.

Many variations of interrogating apparatus 32 are possible. For example,the interrogating apparatus may include one or more antenna arrays 44,such as a transmit array 45 of one or more antenna units, each of whichmay further include a single antenna that transmits radiation 40 or aplurality of antennae that collectively transmit radiation. A receivearray 46 may receive radiation 42 reflected from subject 38. Optionally,some embodiments may employ one or more antennae apparatus as describedin U.S. patent application Ser. No. 10/728,456 filed Dec. 5, 2003,entitled “Millimeter-Wave Active Imaging System”, the disclosure ofwhich is incorporated herein by reference. Optionally, each antenna unitmay both transmit and receive radiation.

Depending on the interrogating apparatus, an imaging system may includean apparatus moving mechanism, not shown, that may move interrogatingapparatus 32 relative to a subject 38, for scanning the subject with oneor more transmit and/or receive arrays.

Interrogating apparatus 32 may be coupled to controller 34. Ascontemplated herein, the controller may include structure and functionsappropriate for generating, routing, processing, transmitting andreceiving millimeter-wave signals to and from the interrogatingapparatus. The controller, in this comprehensive sense, may includemultiplexed switching among individual components of the interrogatingapparatus, transmit electronics, receive electronics, and mechanical,optical, electronic, and logic units. The controller thus may send toand receive from the interrogating apparatus signals 48, such astransmit-related signals 49 and receive-related signals 50,respectively. Signals 48 may include appropriate signals, such ascontrol signals and image-related signals.

Controller 34 may include hardware, software, firmware, or a combinationof these, and may be included in a computer, computer server, or othermicroprocessor-based system capable of performing a sequence of logicoperations. In addition, processing may be distributed with individualportions being implemented in separate system components.

Accordingly, controller 34 may include a processor 52 and a memory 54.Controller components such as output devices, processors, memories andmemory devices, and other components, may be wholly or partlyco-resident in interrogation apparatus 32 or be wholly or partly locatedremotely from the interrogation apparatus.

Processor 52 may process data signals received from the interrogatingapparatus. The processor thus may include hardware, software, firmware,or a combination of these, and may be included in a computer, computerserver, or microprocessor-based system capable of performing a sequenceof logic operations. The processor may be any analog or digitalcomputational device, or combination of devices, such as a computer(s),microprocessor(s), or other logic unit(s) adapted to controlinterrogating a subject and receiving signals 50, and to produce imagedata 56 representative of at least a portion of the subjectinterrogated.

The description that follows is presented largely in terms of displayimages, algorithms, and symbolic representations of operation of databits within a computer memory. Software, firmware, and hardwareencompassing such representations may be configured many different ways,and may be aggregated into a single processor and program with unclearboundaries.

An algorithm is generally considered to be a self-consistent sequence ofsteps leading to a desired result. These steps require physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. When stored, they may be stored in any computer-readablemedium. As a convention, these signals may be referred to as bits,values, elements, symbols, characters, images, data, terms, numbers, orthe like. These and similar terms may be associated with appropriatephysical quantities and are convenient labels applied to thesequantities.

In the present case, the operations may include machine operations thatmay be performed automatically and/or in conjunction with a humanoperator. Useful machines for performing the operations disclosedinclude general-purpose digital computers, microprocessors, or othersimilar devices. The present disclosure also relates to apparatus forperforming these operations. This apparatus may be specially constructedfor the required purposes or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer or other apparatus. In particular, various general-purposemachines may be used with programs in accordance with the teachingsherein, or it may prove more convenient to construct more specializedapparatus to perform the required method steps.

A program or programs embodying the disclosed methods need not reside ina single memory, or even a single machine. Various portions, modules orfeatures of them can reside in separate memories, or even separatemachines. The separate machines may be connected directly, or through anetwork, such as a local access network (LAN), or a global network, suchas what is presently generally known as the Internet. Similarly, themachines need not be collocated with each other.

Image data may include any data or data sets, whether processed,partially processed or unprocessed, or sub-sets of the data, such as:data for a portion of a subject; data that is manipulated in order toidentify information corresponding to one or more given features of asubject; data that is manipulated in order to present, for viewing by anoperator or by another processor, information corresponding to one ormore given features of a subject; or measurements or other informationrelating to a subject that is derived from received signals. Image data56 may be output to one or more output devices 36 coupled to processor52, such as a storage medium or device, communication link, such as anetwork hub, another computer or server, a printer, or directly to adisplay device, such as a video monitor. Processor 52 may also becoupled to receive input signals 58 from an input device 60, such as akeyboard, cursor controller, touch-screen display, another processor, anetwork, or other device, communication link, such as a source ofinformation for operating the system or supplemental informationrelating to a given subject.

In some embodiments, processor 52 may be coupled to memory 54 forstoring data, such as one or more data sets produced by processor 52, oroperating instructions, such as instructions for processing data. Memory54, referred to generally as storage media, may be a single device or acombination of devices, and may be local to the processor or remote fromit and accessible on a communication link or network. Operatinginstructions or code may be stored in memory 54, along with image data,and may be embodied as hardware, firmware, or software.

Data produced or accessed by the processor may thus be sent to andretrieved from memory 54 for storage. In some examples, data producedfrom interrogating a given subject or input from another source may beretrieved for further processing, such as identifying informationcorresponding to a feature of the subject, for modifying image data, orfor producing an image of a subject or portion of a subject derived fromreceived signals. In such examples, the processor may be configured toidentify or compare information corresponding to the features, such asidentification of image features obtained from data produced byradiation in different frequency ranges, or at different depths relativeto a reference.

An imaging system, such as that illustrated in FIG. 1, may be used forinterrogating in a variety of applications in which the controller maybe configured to identify and/or process information in one or more datasets corresponding to one or more features of a subject, such as imagedata relating to a subject. A second example of an imaging system 30 isillustrated in FIG. 2. In imaging system 30, a subject 38 in a subjectposition 70 may include a person 72 presented for interrogation bysystem 30. Person 72 is shown wearing clothing 74 over her or his body76, which clothing conceals an object 78, shown in the form of a weapon.Subject 38 may be positioned in an interrogation station or portal 80 ofsystem 30. Portal 80 may be configured in various ways for placement ata security checkpoint where it is desired to detect objects, such asweapons or contraband, on a person or other subject. Portal 80 mayinclude, for example, a platform 82 connected to a motion mechanism 84in the form of motor 86. Platform 82 may be arranged to support subject38. Motor 86 may be arranged to rotate selectively the platform aboutrotational axis R while subject 38 is positioned thereon. For theconfiguration shown, axis R may be vertical, and subject 38 may be in agenerally central subject position 70 relative to axis R and platform82.

Interrogating apparatus 32 may include an antenna apparatus 88 includinga primary multiple-element sensing array 44. The interrogating apparatus32 may include a frame 90 on which array 44 is supported. Array 44 mayextend the full height of frame 40. Motor 38 may cause platform 82, andsubject 38 to rotate about axis R. As a result, array 34 circumscribes agenerally circular pathway about axis R. The antenna array may bepositioned relative to the subject as is appropriate. In some examples,the antenna array is about 0.5 to about 2 meters from the subjectposition.

In this example, antenna array 44 may include a number of linearlyarranged antenna elements 92 only a few of which are schematicallyillustrated. Each element 92 may be dedicated to transmission orreception of radiation or both, and the elements may be arranged in twogenerally vertical columns, with one column dedicated to transmission,and the other to reception. The number and spacing of the elementscorresponds to the wavelengths used and the resolution desired. A rangeof 200 to about 600 elements can span a vertical length of about two ortwo and one-half meters.

Various other configurations for portal 80 and interrogating apparatus32 may be used. For example, a two-dimensional transmit and receivearray may be used, as well as an array that moves around a fixed subjectplatform, or an array that moves vertically and extends horizontally.Further, many variations of an antenna apparatus are possible. Theantenna apparatus may include one or more antenna units, and eachantenna unit may include one or more transmitting antennae and/or one ormore receiving antennae. An antenna unit may include a plurality ofantennae that may receive radiation in response to transmission by asingle antenna. The antennae may be any appropriate type configured totransmit or receive electromagnetic radiation, such as a slot line,patch, endfire, waveguide, dipole, semiconductor, or laser. Antennae mayboth transmit and receive. The antenna units may have one or moreindividual antennae that transmit or receive like polarization or unlikepolarized waveforms, such as plane, elliptical, or circularpolarization, and may have narrow or broad angular radiation beampatterns, depending on the application. Beam width may be relativelybroad, i.e. 30 to 120 degrees for imaging applications that useholographic techniques, while narrow beam widths in the range of 0 to 30degrees may be used for applications having a narrow field of viewrequirement.

Further, a single antenna may scan a subject by mechanically movingabout the subject in a one- or two-dimensional path. A one- ortwo-dimensional array of antenna units may electronically andmechanically scan a subject. An interrogating apparatus may include oneor a plurality of transmit and/or receive antenna apparatus. Theantennae apparatus may be protected from the environment by suitableradome material, which may be part of the apparatus, or separate,depending on the mechanical motion that is required of the antennaeapparatus or array. Examples of other array configurations areillustrated in copending U.S. patent application Ser. No. 10/728,456.

A controller 34 may control operation of interrogating apparatus 32.Controller 34 may include a transceiver 94 including a switching tree 96configured to irradiate subject 38 with only one transmitting element 92at a time, and simultaneously receive with one or more elements 92.Transceiver 94 may include logic to direct successive activation of eachcombination of transmit and receive antenna elements to provide a scanof a portion of a subject 38 along a vertical direction as platform 82and the subject rotate. Other configurations of transceiver 94 may beused. For example, the transceiver may include structurally and/orelectrically separate transmitter(s) and receiver(s).

An image signal 50 received from array 44 may be downshifted infrequency and converted into an appropriate format for processing. Inone form, transceiver 94 may be of a bi-static heterodyne FrequencyModulated Continuous Wave (FM/CW) type like that described in U.S. Pat.No. 5,859,609. Other examples are described in U.S. Pat. Nos. 5,557,283and 5,455,590. In other embodiments, a mixture of different transceiverand sensing element configurations with overlapping or non-overlappingfrequency ranges may be utilized, and may include one or more of theimpulse type, monostable homodyne type, bi-static heterodyne type,and/or other appropriate type.

Transceiver 94 may provide image data 97 corresponding to the imagesignals to one or more processors 52. Processor 52 may include anysuitable component for processing the image data, as appropriate.Processor 52 may be coupled to a memory 54 of an appropriate type andnumber. Memory 54 may include a removable memory device (R.M.D.) 98,such as a tape cartridge, floppy disk, CD-ROM, or the like, as well asother types of memory devices.

Controller 34 may be coupled to motor 86 or other drive element used tocontrol selectively the rotation of platform 82. Controller 34 may behoused in a monitor and control station 100 that may also include one ormore input devices 60, such as operator or network input devices, andone or more displays or other output devices 36.

General Imaging Method

A general flow chart 102, illustrating exemplary operation ofsurveillance system 30, is shown in FIG. 3. Two data acquisition phasesare illustrated. Interrogating apparatus 32 interrogates a subject 38 at104. Image information is detected during the interrogating and an imagesignal is produced. Processor 52 acquires the image signal at 106. Theacquired image signal is then processed at 108 to construct image data.Image data is analyzed to identify image features at 110. As isexplained further below, image features or characteristics may be anyidentifiable aspect of the image data or associated image, such as themagnitude, arrangement, or configuration of data, the shape,configuration, arrangement, texture, location of one or more objects 78relative to a person's body 76, and/or features of the person's body,such as orientation, position, texture, specific body parts, size,shape, configuration, symmetry, or other appropriate aspect.

Where one or more input devices 60 is a source of subject informationseparate from an interrogating apparatus, such as a data base withinformation on a particular person, the data from a supplemental sourcemay be acquired at 112. A supplemental source also may be a sensor thatdetects general features of the subject 38, such as the generaldetection of a substance, a feature identifying the person 72, orcontext data stored in a memory relating to the subject. Suchsupplemental image features may also be identified at 110. The existenceof a substance, an identification of the person or a characteristic,class or categorization of the person, and other appropriate indicatorsor information may be features of the subject, in addition to featuresidentified from the image data.

The various identified image features may then be correlated with eachother at 114. For example, the identification of an object on the sideof a person from an imaging apparatus may be correlated with thedetection of metal in the middle zone of the person, a badge identifyingthe person, and context data previously stored in memory indicating thatthe person is a security guard and has high security clearance.

The identified or correlated features may then be classified at 116.Classification of features may be a logical process for determining thelikelihood that detected features correspond to a suspect object or afalse alarm. For example, the detection of various characteristics orcertain combinations of characteristics in the same zone of an image mayindicate that the image portion is an object. Further, given that theperson is identified as a security guard, it is highly likely that theobject is a gun. Also, the person may be authorized to carry a gun inthis position as part of her duties. The object would thus be given ahigh weight of being a suspect object, but given a low weight as asecurity risk, due to the status of the person as a security guard.

Any set of corresponding features can be assigned a correspondingrelative indicator, such as weight, value or attribute. An area of asubject may thus be assigned a high value even though no image object isdetected. For example, a sheet of plastic explosive taped to a person'sbody may appear smoother than the rest of the person's body. Thestructure of an object also may be the basis of assigning a value, suchas dimensions, shape and edge characteristics.

Once the image features are classified, then conclusions are produced at118 about the combinations of image features. The conclusions may thenbe output at 120, as appropriate, such as via a display, report or alarmcondition.

Use of Images in the Figures

The remaining figures include images that illustrate various exemplaryprocedures and/or features of image data received or produced by aprocessor, such as data received from an interrogating apparatus 32 of asubject. Generally, these images represent data. The steps described maybe performed without actually producing a displayed image, or withoutproducing data that provides visual characteristics suitable fordisplay. An image may be formed as a matrix of picture elements, alsoreferred to as pixels, or simply pels. The images in the figures areintended to facilitate an understanding of the processes described, andare not necessarily a part of the associated process. In FIG. 4, animage 110 of a subject 28 may include portions of the image that relateto the subject and portions of the image that relate to the background,including structure other than the subject. A distinction between thebackground and subject may be provided in an image by variation in avalue of a feature, such as intensity, color and/or a topographicaldata, such as depth or distance from a reference.

Most of the graphical images included in the figures are shown in areverse image format (intensity values are reversed) in order to producelighter images. In the original images, the background is dark and thesubject is light, similar to an expected visual view of a lightedsubject before a dark background. Lighter images tend to be moreaccurately reproduced using such duplicating equipment as printers,copiers and scanners. Thus, although images in which subjects are shownwith lighter, and therefore brighter, intensities may be more readilyand realistically perceived, it will be appreciated that the methodsdisclosed and discussed apply to either form of representation, or toany representation or valuation of data or characteristic that providesa distinction, whether or not suitable for display.

In the example shown in FIG. 4, image 122 includes a relatively lightbackground portion 124 and a darker subject portion 126. Accordingly, inthis representation format, the background generally has an intensitylevel above (lighter than) a threshold, and the subject generally has anintensity level below the threshold.

Due to the nature of the particular interrogating apparatus 32 used toproduce the image data used in the examples disclosed herein, there maybe inconsistencies or anomalies in portions of the image where thebackground has intensity levels similar to those of the subject, and thesubject has intensity levels similar to the background. Image datacorresponding to image 122 may be analyzed to determine features of thesubject, such as the location of the body or a body part, or thelocation of any objects included in the subject. This analysis mayinclude or be based in whole or in part on an analysis of anyappropriate feature or features of the data, such as the intensitylevels of the pixels. Determination of a selected aspect of the imagemay include one or more features of the data.

Frequency-Based Imaging

FIGS. 4–10 illustrate an example of a method for relating imagedifferences based on frequency of transmitted electromagnetic radiation.Different surfaces reflect electromagnetic energy to different degrees.Further, a particular surface may reflect electromagnetic energy ofdifferent frequencies differently. For example, FIG. 4 illustrates animage 122 produced based upon reflectivity of electromagnetic radiationtransmitted onto a subject 38 over a frequency band of 24 GHz to 30 GHz.Signals representative of the reflectivity or reflectance of thetransmitted radiation for different depths of the subject are receivedfrom interrogating apparatus 32. Generally, the intensity values of theimage indicate a subject portion 124 and a background portion 126. Theterm depth is used to refer to the relative distance from a referenceplane, such as corresponding or related to a position of the antennaelements 92. For example, FIG. 2 illustrates a position on subject 38that may have a physical distance or depth D1 from the antenna elements.Image 122 is formed of picture elements, or pixels, that haveintensities determined from the radiation reflected from the subject.The image is produced by selecting from the signals for all of thefrequencies transmitted, the maximum reflectivity for each pictureelement.

Image 122 may be considered a more detailed image overall, since it isbased on the data for received radiation for all transmittedfrequencies. However, it may be useful to relate data received fordifferent frequencies or frequency bands, in the event that a person maybe carrying, or otherwise supporting an object that reflectselectromagnetic radiation of different frequencies differently than doesthe person's body. This may be achieved in a method of surveilling asubject including irradiating at least a portion of the subject withelectromagnetic radiation in at least first and second differentfrequency ranges having frequencies that are more than about 100 MHz andless than about 2 THz; receiving the irradiated electromagneticradiation reflected from the subject; producing, from the receivedelectromagnetic radiation, first image data representative of a firstimage of at least the portion of the subject based at least in part onreflectivity of the electromagnetic radiation in the first frequencyrange; producing, from the received electromagnetic radiation, at leastsecond image data representative of at least a second image of at leastthe portion of the subject based at least in part on reflectivity of theelectromagnetic radiation over at least the second frequency range; andrelating at least the first and second image data.

In one example of performing such a method, image date for a pluralityof frequency ranges is produced, with a frequency range having one ormore frequencies. FIG. 5 illustrates an image 128 including a generallylight background portion 130 and a generally dark subject portion 132,for radiation in a frequency band of 24–26 GHz. Similarly, FIG. 6illustrates an image 134 including a generally light background portion136 and a generally dark subject portion 138, for radiation in afrequency band of 26.1–28 GHz, and FIG. 7 illustrates an image 140including a generally light background portion 142 and a generally darksubject portion 144, for radiation in a frequency band of 28.1–30 GHz.

It is seen that images 128, 134 and 140 are different from each other,and different from image 122. As it turns out, the person in the subjectscanned has an object 78 supported on the side below the right arm. Thisobject takes on different appearances relative to the person's body 76in each of the figures. Such differences may be useful is identifyingthe object, which identification may be performed by data analysis or byevaluation by a user observing displays of the images. In order toenhance the observation of differences in the images, each image may beassigned a distinguishing characteristic, such as an image pattern,intensity level or color. Each of images 128, 134 and 140 are grayscaleimages derived from color images. In this example, the pixels in theseimages were assigned the colors of red, green and blue, respectively.This is at least part of the reason that the general intensity levels ofthese images differ.

The different images produced from data for different frequency rangesmay be displayed serially or concurrently to an observer, to assist theobserver in identifying differences between the images. The areas ofsuch differences may correspond to image anomalies, which may includeobjects. Directing an observer's attention to the anomalies may hastenthe identification of objects.

Optionally, an image formed as a composite of the separate images mayalso be used to identify differences between the images. For example, animage may be formed by displaying each of the image characteristicssimultaneously. In the case of color images, the colors may be combinedto form different ranges of colors. Such an image 146 is illustrated inFIG. 8. As with the previous images, image 146 includes a background 148and a subject 150. Subject 150 corresponds to a person's body 76 and anobject 78. Image 146 was formed by directly adding the colors anddefinition of the individual images 128, 134 and 140. Because each imagewas formed with less information than image 122, it has reduced detaildefinition or clarity.

An image 152 with improved clarity is depicted in FIG. 9. Image 152includes a background 154 and a subject 156. This image consists ofimage 122, being an image of greater detail than image 146, but with theaddition of the colors from image 146. This image was enhanced byincreasing the brightness, contrast, and color saturation. Other imageenhancing techniques may also be used. In regions of this image whereone of the component colors of red, green and blue dominate, it isapparent that this is a region for which the subject produced thehighest level of reflectivity for the three frequency ranges. In regionswhere reflectivity was more balanced between two or three of thefrequency ranges, the image has a color produced by the combination ofcomponent colors. For example, a combination of blue and red colorsappears as purple.

Since color differences are difficult to distinguish in a black andwhite image, generalized and oversimplified regions of image 152 havingdifferent colors are outlined in FIG. 10 to facilitate the visualizationprocess that would be apparent from the original color image. In FIG.10, the letters associated with different regions stand for the firstletter of the corresponding colors of Red, Blue, Green, Orange andPurple. Red, green and blue represent dominant reflectivity in the low,medium and high frequency bands, respectively. It is seen that there area variety of colors making up the image. The central area of the torsois made up of a mix of colors, indicating comparable reflectivity fordifferent frequency bands. The perimeter regions of the torso, though,appear to be dominated by blue along the sides and green in the hip orabdominal region. An exception to this general configuration is a greenregion positioned near where object 78 is located. There are otherlocalized color regions as well. The existence of a limited coloredregion, or region of dominance of reflectivity of one of a plurality offrequency bands, or of a particular frequency band may indicate asuspect area of the image, where closer observation may be warranted, orwhen combined with the classification of other features, may increasethe confidence level in a conclusion that the area is a suspect area.Different frequencies and ranges of frequencies may produce differentresults.

Depth-Based Image Reconstruction

FIGS. 11–18 illustrate a method of producing surveillance images havingreduced depth variance. This may result in images with improvedresolution, clarity and/or depth-related information. In one example, amethod of surveilling a subject may include irradiating at least aportion of the subject with electromagnetic radiation having one or morefrequencies between about 100 MHz and about 2 THz. The irradiatedelectromagnetic radiation reflected from the subject may then bereceived and processed. Processing may include producing, from thereceived electromagnetic radiation, at least first image datarepresentative of at least the portion of the subject based at least inpart on reflectivity of the electromagnetic radiation for a plurality ofdepths for each of a plurality of picture elements or pixels. A firstdepth for each of the plurality of pixels may be selected based at leastin part on the reflectivity of the electromagnetic radiation at at leastthe first depth. A second depth for each of the plurality of pixels maybe selected based at least in part on the first depth of at leastanother one of the pixels. Reflectivity of the electromagnetic radiationat the second depth may be at least part of the basis for producing atleast second image data representative of at least a first image of atleast the portion of the subject.

FIG. 11 illustrates in simplified form what may be considered athree-dimensional data cube 160 corresponding to the organization ofimage data produced as a result of interrogating a subject withelectromagnetic radiation. The X and Y-axes correspond to horizontal andvertical positions on a two-dimensional image. An X-coordinate and aY-coordinate correspond to a pixel, such as pixels 162. The Z-axisrepresents relative depth, or distance from a reference. The image datathen includes a series of cuboids for each pixel having valuescorresponding to a reflectivity for a transmitted electromagneticradiation at that depth. Thus, for each depth z along the Z-axis, a cubeof data space, referred to as a cuboid, volume element or voxel, such asvoxels 164. For each pixel, then, there are a series of associatedvoxels, with each voxel corresponding to a particular depth and having avalue representative of the reflectivity at that depth.

Images may be produced by selecting the maximum reflectivity from theseries of voxels for each pixel. Image 166 of FIG. 12 was produced inthis way. This figure, again with reverses-intensity, includes abackground portion 168 and subject portion 170. The rectangular outlinerepresents a window around an object 78. Image 166 was produced by firstproducing a data cube 160 for each frequency. Then, the maximum value ofeach voxel from the data cubes for all of the frequencies was selectedto form a combined data cube. The maximum values of voxels for each datacube are used to produce a two-dimensional (x, y) image from athree-dimensional (x, y, z) data cube. Since the maximum reflectivityvalue for a given pixel may be for a depth that may vary from pixel topixel, information about the surface of a subject at a given depth maybe lost. For example, if the intensity value in a localized area dropsoff dramatically in the X or Y directions, that may not be apparent froman image based on the maximum reflectivity values.

FIG. 13 depicts an image 172 produced by mapping the z or depth valuesfor the pixels making up image 166. In image 172, the darker theintensity value is for a pixel, the closer it is to the viewer orreference. Conversely, the lighter the value is, the further theposition is from viewer. It is seen that there is a center portion 174,corresponding to the area of subject portion 170 of image 166, havingpixels with generally intermediate intensity values, represented byintermediate shades of gray.

FIG. 14 is a chart showing reflectance or reflectivity as a function ofdepth for three adjacent pixels within the object window shown in image166. The three pixels then have respective sets 175, 176 and 177 ofdepth data. Line 178, connecting data points in set 175 indicated by the“×” symbols, has a maximum reflectance value of about 9.1 at a depth of28. Similarly, line 180, connecting data points in set 176 indicated bythe “*” symbols, has a maximum reflectance value of about 7.3, also at adepth of 28. Line 182, connecting data points in set 177 indicated bythe “+” symbol, has a maximum reflectance value of about 6.4, at a depthof 27.

All three maximum values were used to produce image 166. It can be seenin FIG. 14, that if an image were formed using reflectance values forthe same depth in at least local portions of the image, the image wouldbe different. For example, the values at depth 28 may be selected, inwhich case the reflectance value for the pixel represented by line 182would have been about 3.8 instead of about 6.4, making the change inreflectance, which may be due to an edge of an object, more pronounced.

FIG. 15 is an image 184 produced by using reflectance values existing ata single first depth plane (single z value for all pixels) correspondingto a maximum reflectance in the chart of FIG. 14, such as depth 28.Similarly, FIG. 16 is an image 188 produced by using reflectance valuesexisting at a single second depth corresponding to a maximum reflectancein the chart of FIG. 14, such as depth 27. Object 78 appears moresalient in single depth-plane images 184 and 188, than in image 166having the maximum values across all depth planes. However, parts ofimages 184 and 188 that are at significantly different depths are nolonger visible on these depth planes. For example, the upper portion,corresponding to the head of the person in the subject, is at a furtherdepth, and so tends to not be visible in the selected depth planes.

A more complete image may be realized by localizing the depth plane indifferent portions of an image. One approach for doing this, is to limitthe change in depth planes over localized regions, such as within aselect group associated to a given pixel. The group may be related tothe selected pixel in different ways, such as within a selected distanceor position relative to a selected pixel, or within a selected window orkernel of pixels that may contain the given pixel. In one example, anm×n kernel may be considered, where m and n are integers and the givenpixel is in the kernel, or even in the center of the kernel when oddintegers are selected for the kernel size. Other group selections may bemade, as appropriate in a given application.

Various criteria may be used for making depth plane selection moreconsistent in localized areas. For example, a depth-controlled image maybe produced for an area identified as being suspect based on automaticor manual classification of other features. Optionally, an algorithm maybe applied to an image, such as a set of selection criteria or anumerical selection function, as appropriate.

In some examples, numerical functions may be applied to an imageproduced based on data characteristics, such as maximum reflectancevalues as were used to produce image 166. As a specific example, themedian or mean of a local region of depth values represented by image172 in FIG. 13, may be selected. As another example, a mode may be usedto select a depth value for a given pixel. In a mode selectionalgorithm, a depth value for a selected pixel may be set the same as thedepth value that occurs most frequently or with at least a thresholdfrequency. If no depth value occurs with at least the thresholdfrequency, then a mean, median or other value may be selected.

FIG. 17 is an image 192 of depths selected in a depth filter algorithm.Specifically, image 192 was produced using the median as a filter toapply to the depth data for an image, such as to the depths havingmaximum reflectance values for the pixels as illustrated by image 172 inFIG. 13. This algorithm selects the same or nearly the same depth planefor a pixel as a depth plane of other nearby pixels, thereby expandingthose regions having a common depth plane. Regions characterized by avariety of depth planes may still have a variety of depth planes, albeita reduced variety of depth planes. Visually, it is seen that thelocalized spatial variance is reduced, and general uniformity of depthin localized regions is increased.

FIG. 18 depicts an image 196 produced using the depth values for eachpixel that is depicted in depth image 192. Comparing this image to image166, boundaries at object transitions in depth are more apparent, suchas shown by the object image area, and yet the whole body image isvisible. Further, some of the noise and non-relevant pixels have beenremoved, making the image cleaner. For example, a horizontal line in theupper portion of image 166 is gone, because nearby pixels used differentdepth planes. Although a median filter was used in this example, amodified median filter, mode, or other kinds of linear, nonlinear, orother types of filters may be used, as appropriate.

Depth-Based Imaging

As noted above with reference to images 184 and 188, depicted in FIGS.15 and 16, depth-plane images may provide image information that is notapparent in an image derived from a maximum reflectance value for eachpixel. The relative differences between images at different depths maygive meaningful information about a subject that is depicted in theimages. As further examples of this, FIGS. 19A–21B illustrate differentsingle-plane images from image data used to produce image 122 in FIG. 4.More specifically, FIG. 19A is an image 200, having a background portion202 and a subject portion 204, produced by using reflectance values in asingle first depth plane for all pixels. FIG. 19B is an image 200′illustrating generally and simplistically the areas of higherreflectance as subject portion 204′ characteristic of the subjectportion in the image of FIG. 19A. Similarly, FIG. 20A is an image 206,having a background portion 208 and a subject portion 210, produced byusing reflectance values in a second depth plane. FIG. 20B is an image206′ illustrating generally the areas of higher reflectance as subjectportion 210′ in the image of FIG. 20A. FIG. 21A is an image 212, havinga background portion 214 and a subject portion 216, produced by usingreflectance values in a third depth plane. FIG. 21B is an image 212′illustrating generally the areas of higher reflectance in the image ofFIG. 21A.

It is seen that in this example, each image is different in the area ofobject 78. In image 200, the object is not particularly discernable fromthe person's body. In image 206, the person's body is imaged, but thearea of the object is not well imaged. In image 212, the contraryappears to exist. That is, the object area is more pronounced than theportion of the body immediately surrounding the object. Such featuresmay be apparent when each image is or the images are displayed for anoperator of the imaging system, or may be useful for automaticclassification of these features, such as by comparison of images orcomparison of an image with an expected or reference image.

FIG. 22 is an image 220 formed as a set of side-by-side images 200, 206and 212 of FIGS. 19A, 20A and 21A, respectively. By presentingsequential depth plane images concurrently on a display, as shown, or bydisplaying them serially, such as in the same position, such differencesmay become apparent to an observer. When identified, suspect areas maybe imaged in greater detail, or the suspect areas of the person imagedmay be physically searched to determine the nature of any objects foundin these areas.

Viewing and/or identifying differences between the depth plane imagesmay be further enhanced by assigning them with distinguishingcharacteristics, such as a graphic pattern, intensity level, or color.By assigning the subject of each image with a different color, forinstance, differences between the images can be readily identified bydifferences in the colors. When the colored images are used to form acomposite image that is a combination of the different images, the areasof difference may appear as distinct colors and the areas of similaritymay be a color or colors formed by the combination of the separate imagecolors. FIG. 23A is an image 224 having a background portion 226 and asubject portion 228 formed by combining the images of FIGS. 19A, 20A and21A. In an example in which the subject portions of images 200, 206 and212 are assigned the respective colors of blue, green and red withbrightness corresponding to the respective reflectivity, the torsoappears generally green, the object area and abdominal and hip areaappear generally blue with red undertones, and the head appearsprimarily red with some blue undertones. In this particular example,then, the object area appears very distinct, and would readily beapparent to a system operator or observer viewing the color image, ormay be automatically detected.

Because the distinction between colors is not very apparent in the blackand white image of FIG. 23A, the combination of colors is illustrated inFIG. 23B as a very general and inaccurate combination of images 200′,206′ and 212′ collectively represented as a composite image 224′ havinga mixed subject portion 228′. The overlap of patterned subject portionsillustrate, though, how the different images can produce areas of basecolors (without overlap) and areas of new colors formed by the overlapof base colors, depending on the brightness of the individual colors.

It will be appreciated that the images 200, 206 and 212 are based onreflectance values in a single depth plane or depth value for all pixelsin the images. These depth planes may be adjacent depth planes or depthplanes that are spaced apart by a plurality of incremental depth values.Similar images may also be produced based on a first image made ofreflectance values from different depth values for different pixels. Anexample of this is image 166 of FIG. 12 in which the maximum reflectancevalue is selected for each pixel. One or more images may then also beproduced that are selected depth values spaced from the depth values forwhich reflectance values were selected for the first image. For example,images produced from reflectance values at a distance one or more depthincrements may be used for comparison. Similar approaches may be usedfor producing comparative images based on other first images ofinterest.

Imaging with Increased Precision

As discussed above, image data may be produced by sampling data atdiscrete points for a data cube having discrete voxels or discrete depthvalues for each pixel. As a result, data between the discrete points isnot available. If that data were available, additional image detailwould be available. However, assuming that the information is continuousand varies predictably between discrete data points, increased imageprecision may be provided by estimating any change in data between datapoints. Generally, a method of surveilling a subject may includeirradiating at least a portion of the subject with electromagneticradiation having one or more frequencies between about 100 MHz and about2 THz, receiving the irradiated electromagnetic radiation reflected fromthe subject, producing, from the received electromagnetic radiation, atleast first image data representative of at least the portion of thesubject based at least in part on reflectivity of the electromagneticradiation for a plurality of spaced-apart first depths for each of aplurality of adjacent picture elements, and determining a value ofreflectivity for an intermediate depth between two adjacent firstdepths.

One way that this may be done is to fit the data points to a continuousor discontinuous numerical function. A simple approach is to determinevalues using a rectilinear line containing adjacent discrete datapoints. Values at points on the line between the discrete data pointsmay be used as an estimate of the data at those points. In regions wherethe data is progressively increasing, decreasing or constant, this mayprovide a reasonable approximation of, in this case, reflectivity valuesbetween the data points. However, in regions where the data changesnon-proportionally, this method may produce imprecise values.

Of particular interest in imaging systems using reflectivity ofelectromagnetic radiation for imaging of a subject, the locations andmagnitudes of maximum reflectivity for discrete picture elements areindicative of the location of the surface of the subject. FIG. 24 is achart of reflectance as a function of depth for two representativepicture elements of a data cube resulting from interrogating a subjectwith electromagnetic radiation. In this example, a first set 230 ofreflectance values for different depths are identified by the “+”symbols connected by a line 232. A second set 234 of reflectance valuesfor different depths are identified by the “×” symbols connected by aline 236. The reflectance values for each set are given for eachdiscrete relative depth value between 13 and 20. The maximum values forboth of sets 230 and 234 are at depth value 17. Based strictly on thediscrete values, then, it would appear that the surface of the subjectis at the same depth for the pixels associated with these sets.

Lines 232 and 236 form generally bell-shaped curves, with line 232generally disposed to the left of line 236. This would seem to indicatethat the actual maximum for set 230 may be to the left of the maximumfor set 234. Assuming that reflectance varies continuously as a functionof depth value, an estimate of the location and magnitude of a maximumvalue based on the discrete points may be determined by predicting howthe reflectance varies in the vicinity of the maximum. It has been foundthat a more likely maximum may be determined from each of the sets ofdepth-based values by fitting a polynomial to data points including themaximum data points.

In particular, an inverse parabolic interpolation may be performed basedon the maximum data point and the data point on each side of the maximumdata point. Thus, in this example, the reflectance values for depthvalues of 16, 17 and 18 are used. In the case of set 230 of reflectancevalues, a parabolic curve 238 is determined that passes through thethree data points of set 230. Curve 238 has a maximum value of about12.7 at an intermediate depth value of about 16.6. As might be expected,a parabolic curve 240 containing the three data points of set 234 ofreflectance values is positioned to the right of curve 238. Curve 240has a maximum at a reflectance value of about 11.8 at an intermediatedepth value of about 17.1.

By applying an interpolation algorithm to the set of reflectance valuesfor each of the pixels in an image data cube, different maximum valuesare produced that in turn alter an image produced based on the maximumvalues. In particular, the intensity, brightness, color, or other imagecharacteristic values are changed. The resulting image may be moreprecise than an image based only on discrete values, and may makeobjects held by a person to be more distinguishable. Such an approachmay be applied between other discrete data points, such as x or y-valuesof pixels.

INDUSTRIAL APPLICABILITY

The methods and apparatus described in the present disclosure areapplicable to security, monitoring and other industries in whichsurveillance or imaging systems are utilized.

While embodiments of imaging systems and methods of imaging have beenparticularly shown and described, many variations may be made therein.This disclosure may include one or more independent or interdependentinventions directed to various combinations of features, functions,elements and/or properties, one or more of which may be defined in thefollowing claims. Other combinations and sub-combinations of features,functions, elements and/or properties may be used. Such variations,whether they are directed to different combinations or directed to thesame combinations, whether different, broader, narrower or equal inscope, are also regarded as included within the subject matter of thepresent disclosure. The foregoing embodiments are illustrative, and nosingle feature or element is essential to all possible combinations thatmay be claimed in this or later applications. The claims, accordingly,define inventions disclosed in the foregoing disclosure, but any oneclaim does not necessarily encompass all features or combinations thatmay be claimed.

Where the claims recite “a” or “a first” element or the equivalentthereof, such claims include one or more such elements, neitherrequiring nor excluding two or more such elements. Further, ordinalindicators, such as first, second or third, for identified elements areused to distinguish between the elements, and do not indicate a requiredor limited number of such elements, and do not indicate a particularposition or order of such elements unless otherwise specifically stated.

1. A method of surveilling a subject comprising: irradiating at least aportion of the subject with electromagnetic radiation having one or morefrequencies between about 100 MHz and about 2 THz; receiving theirradiated electromagnetic radiation reflected from the subject;producing, from the received electromagnetic radiation, at least firstimage data representative of at least the portion of the subject basedat least in part on reflectivity of the electromagnetic radiation for aplurality of depths for each of a plurality of picture elements;selecting a first depth for each of the plurality of picture elementsbased at least in part on the reflectivity of the electromagneticradiation at at least the first depth; selecting a second depth for eachof the plurality of picture elements based at least in part on the firstdepth of at least another one of the picture elements; and producing atleast second image data representative of at least a first image of atleast the portion of the subject based at least in part on reflectivityof the electromagnetic radiation at the second depth.
 2. The method ofclaim 1, in which selecting a first depth includes selecting a firstdepth corresponding to a depth at which the reflectivity is a maximumfor the corresponding picture element.
 3. The method of claim 1, inwhich selecting a second depth includes selecting a second depth from aplurality of depths including the first depth.
 4. The method of claim 1,in which selecting a second depth of one picture element is based atleast in part on the first depth of each of a group of picture elementsincluding the one picture element.
 5. The method of claim 4, in whichthe group is an m×n kernel of adjacent picture elements, where m and nare integers.
 6. The method of claim 4, in which selecting a seconddepth of one picture element includes performing a numerical function onthe depths of the picture elements in the group.
 7. The method of claim6, in which performing a numerical function includes computing at leastone of a median, a mode, and a mean.
 8. The method of claim 1, in whichselecting a second depth includes selecting a second depth in a mannertending to make the second depths have reduced variability betweenadjacent picture elements compared to the first depths.
 9. An imagingsystem comprising: an interrogating apparatus configured to transmittoward and receive from a subject in a subject position, electromagneticradiation having one or more frequencies between about 100 MHz and about2 THz, the interrogating apparatus producing an image signalrepresentative of the received radiation; and a controller adapted toproduce, from the received electromagnetic radiation, at least firstimage data representative of at least the portion of the subject basedat least in part on reflectivity of the electromagnetic radiation for aplurality of depths for each of a plurality of picture elements, toselect a first depth for each of the plurality of picture elements basedat least in part on the reflectivity of the electromagnetic radiation atat least the first depth, to select a second depth for each of theplurality of picture elements based at least in part on the first depthof at least another one of the picture elements, and to produce at leastsecond image data representative of at least a first image of at leastthe portion of the subject based at least in part on reflectivity of theelectromagnetic radiation at the second depth.
 10. The system of claim9, in which the controller is further adapted to select a first depthcorresponding to a depth at which the reflectivity is a maximum for thecorresponding picture element.
 11. The system of claim 9, in which thecontroller is further adapted to select a second depth from a pluralityof depths including the first depth.
 12. The system of claim 9, in whichthe controller is further adapted to select a second depth of onepicture element based at least in part on the first depth of each of agroup of picture elements including the one picture element.
 13. Thesystem of claim 12, in which the group is an m×n kernel of adjacentpicture elements, where m and n are integers.
 14. The system of claim12, in which the controller is further adapted to perform a numericalfunction on the depths of the picture elements in the group.
 15. Thesystem of claim 14, in which the controller is further adapted tocompute at least one of a median, a mode, and a mean.
 16. The system ofclaim 9, in which the controller is further adapted to select a seconddepth in a manner tending to make the second depths have reducedvariability between adjacent picture elements compared to the firstdepths.
 17. The system of claim 9, in which the controller is furtheradapted to produce image data corresponding to a three-dimensionalholographic image of at least a portion of the subject.
 18. One or morestorage media having embodied therein a program of commands adapted tobe executed by a computer processor to: irradiate at least a portion ofthe subject with electromagnetic radiation having one or morefrequencies between about 100 MHz and about 2 THz; receive theirradiated electromagnetic radiation reflected from the subject;produce, from the received electromagnetic radiation, at least firstimage data representative of at least the portion of the subject basedat least in part on reflectivity of the electromagnetic radiation for aplurality of depths for each of a plurality of picture elements; selecta first depth for each of the plurality of picture elements based atleast in part on the reflectivity of the electromagnetic radiation at atleast the first depth; select a second depth for each of the pluralityof picture elements based at least in part on the first depth of atleast another one of the picture elements; and produce at least secondimage data representative of at least a first image of at least theportion of the subject based at least in part on reflectivity of theelectromagnetic radiation at the second depth.